Natural statistics of head roll: implications for Bayesian inference in spatial orientation.

We previously proposed a Bayesian model of multisensory integration in spatial orientation (Clemens IAH, de Vrijer M, Selen LPJ, van Gisbergen JAM, Medendorp WP. J Neurosci 31: 5365-5377, 2011). Using a Gaussian prior, centered on an upright head orientation, this model could explain various perceptual observations in roll-tilted participants, such as the subjective visual vertical, the subjective body tilt (Clemens IAH, de Vrijer M, Selen LPJ, van Gisbergen JAM, Medendorp WP. J Neurosci 31: 5365-5377, 2011), the rod-and-frame effect (Alberts BBGT, de Brouwer AJ, Selen LPJ, Medendorp WP. eNeuro 3: ENEURO.0093-16.2016, 2016), as well as their clinical (Alberts BBGT, Selen LPJ, Verhagen WIM, Medendorp WP. Physiol Rep 3: e12385, 2015) and age-related deficits (Alberts BBGT, Selen LPJ, Medendorp WP. J Neurophysiol 121: 1279-1288, 2019). Because it is generally assumed that the prior reflects an accumulated history of previous head orientations, and recent work on natural head motion suggests non-Gaussian statistics, we examined how the model would perform with a non-Gaussian prior. In the present study, we first experimentally generalized the previous observations in showing that also the natural statistics of head orientation are characterized by long tails, best quantified as a t-location-scale distribution. Next, we compared the performance of the Bayesian model and various model variants using such a t-distributed prior to the original model with the Gaussian prior on their accounts of previously published data of the subjective visual vertical and subjective body tilt tasks. All of these variants performed substantially worse than the original model, suggesting a special value of the Gaussian prior. We provide computational and neurophysiological reasons for the implementation of such a prior, in terms of its associated precision-accuracy trade-off in vertical perception across the tilt range.NEW & NOTEWORTHY It has been argued that the brain uses Bayesian computations to process multiple sensory cues in vertical perception, including a prior centered on upright head orientation which is usually taken to be Gaussian. Here, we show that non-Gaussian prior distributions, although more akin to the statistics of head orientation during natural activities, provide a much worse explanation of such perceptual observations than a Gaussian prior.

Predictive steering: integration of artificial motor signals in self-motion estimation.

The brain's computations for active and passive self-motion estimation can be unified with a single model that optimally combines vestibular and visual signals with sensory predictions based on efference copies. It is unknown whether this theoretical framework also applies to the integration of artificial motor signals, such as those that occur when driving a car, or whether self-motion estimation in this situation relies on sole feedback control. Here, we examined if training humans to control a self-motion platform leads to the construction of an accurate internal model of the mapping between the steering movement and the vestibular reafference. Participants (n = 15) sat on a linear motion platform and actively controlled the platform's velocity using a steering wheel to translate their body to a memorized visual target (motion condition). We compared their steering behavior to that of participants (n = 15) who remained stationary and instead aligned a nonvisible line with the target (stationary condition). To probe learning, the gain between the steering wheel angle and the platform or line velocity changed abruptly twice during the experiment. These gain changes were virtually undetectable in the displacement error in the motion condition, whereas clear deviations were observed in the stationary condition, showing that participants in the motion condition made within-trial changes to their steering behavior. We conclude that vestibular feedback allows not only the online control of steering but also a rapid adaptation to the gain changes to update the brain's internal model of the mapping between the steering movement and the vestibular reafference.NEW & NOTEWORTHY Perception of self-motion is known to depend on the integration of sensory signals and, when the motion is self-generated, the predicted sensory reafference based on motor efference copies. Here we show, using a closed-loop steering experiment with a direct coupling between the steering movement and the vestibular self-motion feedback, that humans are also able to integrate artificial motor signals, like the motor signals that occur when driving a car.

Even well-practiced movements benefit from repetition.

Professional golfers spend years practicing, but will still perform one or two practice swings without a ball before executing the actual swing. Why do they do this? In this study, we tested the hypothesis that repeating a well-practiced movement leads to a reduction of movement variability. To operationalize this hypothesis, participants were tested in a center-out reaching task with four different targets, on four different days. To probe the effect of repetition they performed random sequences from one to six movements to the same target. Our findings show that, with repetition, movements are not only initiated earlier but their variability is reduced across the entire movement trajectory. Furthermore, this effect is present within and across the four sessions. Together, our results suggest that movement repetition changes the tradeoff between movement initiation and movement precision.NEW & NOTEWORTHY Professional athletes practice movements that they have performed thousands of times in training just before it is their turn in a game. Why do they do this? Our results indicate that both initial and endpoint variability reduce with repetition in a short sequence of reaching movements. This means that even well-practiced movements benefit from practice.

A neural surveyor to map touch on the body.

Perhaps the most recognizable sensory map in all of neuroscience is the somatosensory homunculus. Although it seems straightforward, this simple representation belies the complex link between an activation in a somatotopic map and the associated touch location on the body. Any isolated activation is spatially ambiguous without a neural decoder that can read its position within the entire map, but how this is computed by neural networks is unknown. We propose that the somatosensory system implements multilateration, a common computation used by surveying and global positioning systems to localize objects. Specifically, to decode touch location on the body, multilateration estimates the relative distance between the afferent input and the boundaries of a body part (e.g., the joints of a limb). We show that a simple feedforward neural network, which captures several fundamental receptive field properties of cortical somatosensory neurons, can implement a Bayes-optimal multilateral computation. Simulations demonstrated that this decoder produced a pattern of localization variability between two boundaries that was unique to multilateration. Finally, we identify this computational signature of multilateration in actual psychophysical experiments, suggesting that it is a candidate computational mechanism underlying tactile localization.

Sensitivity to error during visuomotor adaptation is similarly modulated by abrupt, gradual, and random perturbation schedules.

It has been suggested that sensorimotor adaptation involves at least two processes (i.e., fast and slow) that differ in retention and error sensitivity. Previous work has shown that repeated exposure to an abrupt force field perturbation results in greater error sensitivity for both the fast and slow processes. Although this implies that the faster relearning is associated with increased error sensitivity, it remains unclear what aspects of prior experience modulate error sensitivity. In the present study, we manipulated the initial training using different perturbation schedules, thought to differentially affect fast and slow learning processes based on error magnitude, and then observed what effect prior learning had on subsequent adaptation. During initial training of a visuomotor rotation task, we exposed three groups of participants to either an abrupt, a gradual, or a random perturbation schedule. During a testing session, all three groups were subsequently exposed to an abrupt perturbation schedule. Comparing the two sessions of the control group who experienced repetition of the same perturbation, we found an increased error sensitivity for both processes. We found that the error sensitivity was increased for both the fast and slow processes, with no reliable changes in the retention, for both the gradual and structural learning groups when compared to the first session of the control group. We discuss the findings in the context of how fast and slow learning processes respond to a history of errors.NEW & NOTEWORTHY We investigated what aspects of prior experience modulate error sensitivity, within the framework of a two-state model of short-term sensorimotor adaptation. We manipulated initial training on a visuomotor adaptation reaching task using specific perturbation schedules that are thought to differentially affect fast and slow learning processes, and we tested what effect these had on subsequent adaptation. We found that sensitivity to adaptation error was similarly modulated by abrupt, gradual, and random perturbation schedules.

Movement preparation time determines movement variability.

Faster movements are typically more variable-a speed-accuracy trade-off known as Fitts' law. Are movements that are initiated faster also more variable? Neurophysiological work has associated larger neural variability during motor preparation with longer reaction time (RT) and larger movement variability, implying that movement variability decreases with increasing RT. Here, we recorded over 30,000 reaching movements in 11 human participants who moved to visually cued targets. Half of the visual cues were accompanied by a beep to evoke a wide RT range in each participant. Results show that initial reach variability decreases with increasing RT, for voluntarily produced RTs up to ∼300 ms, whereas other kinematic aspects and endpoint accuracy remained unaffected. We conclude that movement preparation time determines initial movement variability. We suggest that the chosen movement preparation time reflects a trade-off between movement initiation and precision.NEW & NOTEWORTHY Fitts' law describes the speed-accuracy trade-off in the execution of human movements. We examined whether there is also a trade-off between movement planning time and initial movement precision. We show that shorter reaction times result in higher initial movement variability. In other words, movement preparation time determines movement variability.

Prediction and final temporal errors are used for trial-to-trial motor corrections.

Many daily life situations (e.g. dodging an approaching object or hitting a moving target) require people to correct planning of future movements based on previous temporal errors. However, the actual temporal error can be difficult to perceive: imagine a baseball batter that swings and misses a fastball. Here we show that in such situations people can use an internal error signal to make corrections in the next trial. This signal is based on the discrepancy between the actual and the planned action onset time: the prediction error. In this study, we used three interception tasks: reaching movements, saccadic eye movements and a button press that released a cursor moving ballistically for a fixed time. We found that action onset depended on the previous temporal error in the arm movement experiment only and not in the saccadic and button press experiments. However, this dependency was modulated by the movement time: faster arm movements depended less on the previous actual temporal error. An analysis using a Kalman filter confirmed that people used the prediction error rather than the previous temporal error for trial-by-trial corrections in fast arm movements, saccades and button press.

Vestibular modulation of visuomotor feedback gains in reaching.

Humans quickly and sophisticatedly correct their movements in response to changes in the world, such as when reaching to a target that abruptly changes its location. The vigor of these movement corrections is time-dependent, increasing if the time left to make the correction decreases, which can be explained by optimal feedback control (OFC) theory as an increase of optimal feedback gains. It is unknown whether corrections for changes in the world are as sophisticated under full-body motion. For successful visually probed motor corrections during full-body motion, not only the motion of the hand relative to the body needs to be taken into account, but also the motion of the hand in the world should be considered, because their relative positions are changing. Here, in two experiments, we show that visuomotor feedback corrections in response to target jumps are more vigorous for faster passive full-body translational acceleration than for slower acceleration, suggesting that vestibular information modulates visuomotor feedback gains. Interestingly, these corrections do not demonstrate the time-dependent characteristics that body-stationary visuomotor feedback gains typically show, such that an optimal feedback control model fell short to explain them. We further show that the vigor of corrections generally decreased over the course of trials within the experiment, suggesting that the sensorimotor system adjusted its gains when learning to integrate the vestibular input into hand motor control.NEW & NOTEWORTHY Vestibular information is used in the control of reaching movements to world-stationary visual targets, while the body moves. Here, we show that vestibular information also modulates the corrective reach responses when the target changes position during the body motion: visuomotor feedback gains increase for faster body acceleration. Our results suggest that vestibular information is integrated into fast visuomotor control of reaching movements.

Bayesian adaptive stimulus selection for dissociating models of psychophysical data.

Comparing models facilitates testing different hypotheses regarding the computational basis of perception and action. Effective model comparison requires stimuli for which models make different predictions. Typically, experiments use a predetermined set of stimuli or sample stimuli randomly. Both methods have limitations; a predetermined set may not contain stimuli that dissociate the models, whereas random sampling may be inefficient. To overcome these limitations, we expanded the psi-algorithm (Kontsevich & Tyler, 1999) from estimating the parameters of a psychometric curve to distinguishing models. To test our algorithm, we applied it to two distinct problems. First, we investigated dissociating sensory noise models. We simulated ideal observers with different noise models performing a two-alternative forced-choice task. Stimuli were selected randomly or using our algorithm. We found using our algorithm improved the accuracy of model comparison. We also validated the algorithm in subjects by inferring which noise model underlies speed perception. Our algorithm converged quickly to the model previously proposed (Stocker & Simoncelli, 2006), whereas if stimuli were selected randomly, model probabilities separated slower and sometimes supported alternative models. Second, we applied our algorithm to a different problem-comparing models of target selection under body acceleration. Previous work found target choice preference is modulated by whole body acceleration (Rincon-Gonzalez et al., 2016). However, the effect is subtle, making model comparison difficult. We show that selecting stimuli adaptively could have led to stronger conclusions in model comparison. We conclude that our technique is more efficient and more reliable than current methods of stimulus selection for dissociating models.

Flexible Visuomotor Associations in Touchscreen Control.

To move real objects, our hand needs to get in direct physical contact with the object. However, this is not necessarily the case when interacting with virtual objects, for example when displacing objects on tablets by swipe movements. Here, we performed two experiments to study the behavioral strategies of these movements, examining how visual information about the virtual object is mapped into a swipe that moves the object into a goal location. In the first experiment, we investigated how swiping behavior depends on whether objects were located within or outside the swiping workspace. Results show that participants do not start the swipe movement by placing their finger on the virtual object, as they do when reaching to real objects, but rather keep a systematic distance between the object location and the initial swipe location. This mismatch, which was experimentally imposed by placing the object outside the workspace, also occurred when the object was within the workspace. In the second experiment, we investigated which factors determine this mismatch by systematically manipulating the initial hand location, the location of the object and the location of the goal. Dimensionality reduction of the data showed that three factors are taken into account when participants choose the initial swipe location: the expected total movement distance, the distance between their finger on the screen and the object, and a preference not to cover the object. The weight given to each factor differed among individuals. These results delineate, for the first time, the flexibility of visuomotor associations in the virtual world.

Effect of depth information on multiple-object tracking in three dimensions: A probabilistic perspective.

Many daily situations require us to track multiple objects and people. This ability has traditionally been investigated in observers tracking objects in a plane. This simplification of reality does not address how observers track objects when targets move in three dimensions. Here, we study how observers track multiple objects in 2D and 3D while manipulating the average speed of the objects and the average distance between them. We show that performance declines as speed increases and distance decreases and that overall tracking accuracy is always higher in 3D than in 2D. The effects of distance and dimensionality interact to produce a more than additive improvement in performance during tracking in 3D compared to 2D. We propose an ideal observer model that uses the object dynamics and noisy observations to track the objects. This model provides a good fit to the data and explains the key findings of our experiment as originating from improved inference of object identity by adding the depth dimension.

Decisions in motion: passive body acceleration modulates hand choice.

In everyday life, we frequently have to decide which hand to use for a certain action. It has been suggested that for this decision the brain calculates expected costs based on action values, such as expected biomechanical costs, expected success rate, handedness, and skillfulness. Although these conclusions were based on experiments in stationary subjects, we often act while the body is in motion. We investigated how hand choice is affected by passive body motion, which directly affects the biomechanical costs of the arm movement due to its inertia. With the use of a linear motion platform, 12 right-handed subjects were sinusoidally translated (0.625 and 0.5 Hz). At 8 possible motion phases, they had to reach, using either their left or right hand, to a target presented at 1 of 11 possible locations. We predicted hand choice by calculating the expected biomechanical costs under different assumptions about the future acceleration involved in these computations, being the forthcoming acceleration during the reach, the instantaneous acceleration at target onset, or zero acceleration as if the body were stationary. Although hand choice was generally biased to use of the dominant hand, it also modulated sinusoidally with the motion, with the amplitude of the bias depending on the motion's peak acceleration. The phase of hand choice modulation was consistent with the cost model that took the instantaneous acceleration signal at target onset. This suggests that the brain relies on the bottom-up acceleration signals, and not on predictions about future accelerations, when deciding on hand choice during passive whole body motion.NEW & NOTEWORTHY Decisions of hand choice are a fundamental aspect of human behavior. Whereas these decisions are typically studied in stationary subjects, this study examines hand choice while subjects are in motion. We show that accelerations of the body, which differentially modulate the biomechanical costs of left and right hand movements, are also taken into account when deciding which hand to use for a reach, possibly based on bottom-up processing of the otolith signal.

Visuomotor adaptation: how forgetting keeps us conservative.

Even when provided with feedback after every movement, adaptation levels off before biases are completely removed. Incomplete adaptation has recently been attributed to forgetting: the adaptation is already partially forgotten by the time the next movement is made. Here we test whether this idea is correct. If so, the final level of adaptation is determined by a balance between learning and forgetting. Because we learn from perceived errors, scaling these errors by a magnification factor has the same effect as subjects increasing the amount by which they learn from each error. In contrast, there is no reason to expect scaling the errors to affect forgetting. The magnification factor should therefore influence the balance between learning and forgetting, and thereby the final level of adaptation. We found that adaptation was indeed more complete for larger magnification factors. This supports the idea that incomplete adaptation is caused by part of what has been learnt quickly being forgotten.

Structure learning and the Occam's razor principle: a new view of human function acquisition.

We often encounter pairs of variables in the world whose mutual relationship can be described by a function. After training, human responses closely correspond to these functional relationships. Here we study how humans predict unobserved segments of a function that they have been trained on and we compare how human predictions differ to those made by various function-learning models in the literature. Participants' performance was best predicted by the polynomial functions that generated the observations. Further, participants were able to explicitly report the correct generating function in most cases upon a post-experiment survey. This suggests that humans can abstract functions. To understand how they do so, we modeled human learning using an hierarchical Bayesian framework organized at two levels of abstraction: function learning and parameter learning, and used it to understand the time course of participants' learning as we surreptitiously changed the generating function over time. This Bayesian model selection framework allowed us to analyze the time course of function learning and parameter learning in relative isolation. We found that participants acquired new functions as they changed and even when parameter learning was not completely accurate, the probability that the correct function was learned remained high. Most importantly, we found that humans selected the simplest-fitting function with the highest probability and that they acquired simpler functions faster than more complex ones. Both aspects of this behavior, extent and rate of selection, present evidence that human function learning obeys the Occam's razor principle.

How the required precision influences the way we intercept a moving object.

Do people perform a given motor task differently if it is easy than if it is difficult? To find out, we asked subjects to intercept moving virtual targets by tapping on them with their fingers. We examined how their behaviour depended on the required precision. Everything about the task was the same on all trials except the extent to which the fingertip and target had to overlap for the target to be considered hit. The target disappeared with a sound if it was hit and deflected away from the fingertip if it was missed. In separate sessions, the required precision was varied from being quite lenient about the required overlap to being very demanding. Requiring a higher precision obviously decreased the number of targets that were hit, but it did not reduce the variability in where the subjects tapped with respect to the target. Requiring a higher precision reduced the systematic deviations from landing at the target centre and the lag-one autocorrelation in such deviations, presumably because subjects received information about smaller deviations from hitting the target centre. We found no evidence for lasting effects of training with a certain required precision. All the results can be reproduced with a model in which the precision of individual movements is independent of the required precision, and in which feedback associated with missing the target is used to reduce systematic errors. We conclude that people do not approach this motor task differently when it is easy than when it is difficult.

How the statistics of sequential presentation influence the learning of structure.

Recent work has shown that humans can learn or detect complex dependencies among variables. Even learning a simple dependency involves the identification of an underlying model and the learning of its parameters. This process represents learning a structured problem. We are interested in an empirical assessment of some of the factors that enable humans to learn such a dependency over time. More specifically, we look at how the statistics of the presentation of samples from a given structure influence learning. Participants engage in an experimental task where they are required to predict the timing of a target. At the outset, they are oblivious to the existence of a relationship between the position of a stimulus and the required temporal response to intercept it. Different groups of participants are either presented with a Random Walk where consecutive stimuli were correlated or with stimuli that were uncorrelated over time. We find that the structural relationship implicit in the task is only learned in the conditions where the stimuli are independently drawn. This leads us to believe that humans require rich and independent sampling to learn hidden structures among variables.

What autocorrelation tells us about motor variability: insights from dart throwing.

In sports such as golf and darts it is important that one can produce ballistic movements of an object towards a goal location with as little variability as possible. A factor that influences this variability is the extent to which motor planning is updated from movement to movement based on observed errors. Previous work has shown that for reaching movements, our motor system uses the learning rate (the proportion of an error that is corrected for in the planning of the next movement) that is optimal for minimizing the endpoint variability. Here we examined whether the learning rate is hard-wired and therefore automatically optimal, or whether it is optimized through experience. We compared the performance of experienced dart players and beginners in a dart task. A hallmark of the optimal learning rate is that the lag-1 autocorrelation of movement endpoints is zero. We found that the lag-1 autocorrelation of experienced dart players was near zero, implying a near-optimal learning rate, whereas it was negative for beginners, suggesting a larger than optimal learning rate. We conclude that learning rates for trial-by-trial motor learning are optimized through experience. This study also highlights the usefulness of the lag-1 autocorrelation as an index of performance in studying motor-skill learning.

Sensorimotor priors in nonstationary environments.

In the course of its interaction with the world, the human nervous system must constantly estimate various variables in the surrounding environment. Past research indicates that environmental variables may be represented as probabilistic distributions of a priori information (priors). Priors for environmental variables that do not change much over time have been widely studied. Little is known, however, about how priors develop in environments with nonstationary statistics. We examine whether humans change their reliance on the prior based on recent changes in environmental variance. Through experimentation, we obtain an online estimate of the human sensorimotor prior (prediction) and then compare it to similar online predictions made by various nonadaptive and adaptive models. Simulations show that models that rapidly adapt to nonstationary components in the environments predict the stimuli better than models that do not take the changing statistics of the environment into consideration. We found that adaptive models best predict participants' responses in most cases. However, we find no support for the idea that this is a consequence of increased reliance on recent experience just after the occurrence of a systematic change in the environment.

Random walk of motor planning in task-irrelevant dimensions.

The movements that we make are variable. It is well established that at least a part of this variability is caused by noise in central motor planning. Here, we studied how the random effects of planning noise translate into changes in motor planning. Are the random effects independently added to a constant mean end point, or do they accumulate over movements? To distinguish between these possibilities, we examined repeated, discrete movements in various tasks in which the motor output could be decomposed into a task-relevant and a task-irrelevant component. We found in all tasks that the task-irrelevant component had a positive lag 1 autocorrelation, suggesting that the random effects of planning noise accumulate over movements. In contrast, the task-relevant component always had a lag 1 autocorrelation close to zero, which can be explained by effective trial-by-trial correction of motor planning on the basis of observed motor errors. Accumulation of the effects of planning noise is consistent with current insights into the stochastic nature of synaptic plasticity. It leads to motor exploration, which may subserve motor learning and performance optimization.